Linear Regression and Its Applications

where we extended x to (x0 = 1, x1, x2, . . . , xk). Finding the best parameters w is then referred to as linear regression problem, whereas all other types of relationship between the features and the target fall into a category of nonlinear regression. In either situation, the regression problem can be presented as a probabilistic modeling approach that reduces to parameter estimation; i.e. to an optimization problem with the goal of maximizing or minimizing some performance criterion between target values {yi}ni=1 and predictions {f(xi)} n i=1. We can think of a particular optimization algorithm as the learning or training algorithm.